Accurate Buffer Overflow Detection via Abstract Payload Execution

Static buffer overflow exploits belong to the most feared and frequently launched attacks on todays Internet. These exploits target vulnerabilities in daemon processes which provide important network services. Ever since the buffer overflow hacking technique has reached a broader audience due to the Morris Internet worm in 1988 and the infamous paper by AlephOne in the phrack magazine, new weaknesses in many programs have been discovered and abused. Current intrusion detection systems (IDS) address this problem in different ways. Misuse based systems attempt to detect the signature of known exploits in the payload of the network packets. This can be easily evaded by a skilled intruder as the attack code can be changed, reordered or even partially encrypted. Anomaly based network sensors neglect the packet payload and only analyze bursts of traffic thus missing buffer overflows altogether. Host based anomaly detectors that monitor process behavior can notice a successful exploit but only a-posteriori when it has already been successful. In addition, both anomaly variants suffer from high false positive rates. In this paper we present an approach that accurately detects buffer overflow code in the packet’s payload by concentrating on the sledge of the attack. The sledge is used to increase the changes of a successful intrusion by providing a long code segment that simply moves the program counter towards the immediately following exploit code. Although the intruder has some freedom in shaping the sledge it has to be executable by the processor. We perform abstract execution of the payload to identify such sequences of executable code with virtually no false positives. A prototype implementation of our sensor has been integrated into the Apache web server. We have evaluated the detection rates as well as the performance impact of our proposed system.